Method and system for robotic assembly

ABSTRACT

A method for robotic assembly includes: receiving product data including product structure data and/or product geometry data of a product with a base component and at least one assembly part to be assembled; analyzing the product data to determine robot functions relating to functions of a robot for assembly of the product as determined robot functions; generating a robot program including assembly instructions dependent on the determined robot functions and the product data; and executing the generated robot program so as to identify and/or localize the at least one assembly part and assemble the product.

CROSS-REFERENCE TO PRIOR APPLICATION

This application is a continuation of International Patent ApplicationNo. PCT/EP2019/054905, filed on Feb. 27, 2019, the entire disclosure ofwhich is hereby incorporated by reference herein.

FIELD

The present disclosure relates to a method and a system for roboticassembly.

BACKGROUND

Programming industrial robot applications is still time consuming, andrequires expert knowledge in the application domain and in the roboticsdomain. Attempts to generate robot application programs automatically orsemi-automatically are more on the theoretical level, except thegeneration of robot motion paths based on CAD/CAM data. There are alsomany research work on the generation of assembly graphs, but not used inimplementing intelligent robot functions. On the other hand,sensor-based object localization and object feature detection is widelyused in the robotics domain, but using them for individual applicationsrequired often special knowledge and additional programming orconfiguration effort.

Therefore, there is a need for an improved method for robotic assembly.

SUMMARY

In an embodiment, the present invention provides a method for roboticassembly, comprising: receiving product data comprising productstructure data and/or product geometry data of a product with a basecomponent and at least one assembly part to be assembled; analyzing theproduct data to determine robot functions relating to functions of arobot for assembly of the product as determined robot functions;generating a robot program comprising assembly instructions dependent onthe determined robot functions and the product data; and executing thegenerated robot program so as to identify and/or localize the at leastone assembly part and assemble the product.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be described in even greater detail belowbased on the exemplary figures. The invention is not limited to theexemplary embodiments. Other features and advantages of variousembodiments of the present invention will become apparent by reading thefollowing detailed description with reference to the attached drawingswhich illustrate the following:

FIG. 1 shows a flow chart of a system for robot assembly;

FIG. 2 shows a schematic view of a system for robot assembly; and

FIG. 3 shows a flow chart of a method for robot assembly.

DETAILED DESCRIPTION

In an embodiment, the present invention provides a method for roboticassembly, comprising the steps:

receiving product data, comprising product structure data and/or productgeometry data of a product with a base component and at least oneassembly part to be assembled;

analysing the product data, thereby determining robot functions relatingto functions of a robot for assembly of the product;

generating a robot program comprising assembly instructions dependent onthe determined robot function and the product data; and

executing the generated robot program, thereby identifying and/orlocalizing the at least one assembly part and assembling the product.

Preferably, functions of the robot for assembly of the product comprisepushing, snapping and/or screwing.

The term “assembly instructions”, as used herein, relates toinstructions for sequences of a robot in order to assemble the product.Assembling the product preferably comprises picking up the at least oneassembly part, moving the at least one assembly part to an assemblylocation of the base component and/or connecting the at least oneassembly part with the base component.

Preferably, dependent on the structure data, product hierarchy data isdetermined, relating to the hierarchy of the individual members of theproduct. Dependent on the hierarchy of the individual members of theproduct preferably the base component can be distinguished from theassembly parts.

When analysing the product data, the base component and the at least oneassembly part is determined from the product data, preferably from theproduct hierarchy data.

Preferably, the product data comprises data, which is relative to themembers of the product. For example, the product data comprises locationdata of the members of the product relative to another. The product datapreferably comprises workspace data, relating to design and/or layout ofa workspace for assembling the product. The product data may comprisefurther data about the product and/or the product members.

Preferably, the robot program is generated fully automatically.

Thus, an improved method for combining assembly instruction generationwith sensor-based object detection and localization for efficient andautomatic identification of the at least one assembly part andlocalization of the at least one assembly part is provided.

Thus, the effort for programming robotic assembly applications isminimized.

In a preferred embodiment, the method comprises the step:

receiving product data from CAD data of the product.

The CAD data of the product preferably comprises 2D models and/or 3Dmodels of the product and/or members of the product like assembly partsor base component.

CAD systems support definition and analysis of assembly sequences.Preferably, at least product hierarchy data based on product structuredata and product geometry data can be determined based on the CAD data.

Thus, an improved method for combining CAD based assembly instructiongeneration with sensor-based object detection and localization forefficient and automatic identification of the at least one assembly partand localization of the at least one assembly part is provided.

In a preferred embodiment, the method comprises the step:

identifying the at least one assembly part of the product by a RFIDreader while executing the robot program.

The assembly parts are preferably provided with RFID transponders, atleast storing a part identification number of the respective assemblypart. The RFID reader preferably reads out the RFID transponders,thereby identifying the at least one assembly part.

Thus, an improved identification of the at least one assembly part canbe provided.

In a preferred embodiment, the method comprises the step:

localizing the at least one assembly part of the product by a visionsystem while executing the robot program.

Thus, an improved localization of the at least one assembly part can beprovided.

In a preferred embodiment, the method comprises the steps:

determining a CAD model of the at least one assembly part from theproduct data; and

matching a 3D image of the vision system to the determined CAD model ofthe at least one assembly part.

Thus, an improved localization of the at least one assembly part can beprovided

In a preferred embodiment, analysing the product data comprises creatinga local coordinate system for the base component, creating a localcoordinate system for the at least one assembly part, determining astart pose of the at least one assembly part, determining a direction ofthe at least one assembly part and/or determining a picking pose of theat least one assembly part.

Preferably, the start pose of the at least one assembly part, thedirection of the at least one assembly part and/or the picking pose ofthe at least one assembly part are specifications relative to oneanother and/or relative to the base component. The start pose and/or thedirection preferably relate to an assembly motion of the robot.

Thus, an improved method for combining assembly instruction generationwith sensor-based object detection and localization for efficient andautomatic identification of the at least one assembly part andlocalization of the at least one assembly part is provided.

In a preferred embodiment, the assembly instructions comprise findingthe at least one assembly part, adjusting a reference coordinate systemto the local coordinate system of the base component, adjusting thepicking pose of the at least one assembly part, picking the at least oneassembly part with the corresponding picking pose, assembling the atleast one assembly part with the corresponding start pose and directionof the at least one assembly part and/or updating the product data.

Based on the relative data, when the base component is found, the localcoordinate system for the base component and the local coordinate systemfor the at least one assembly part are adjusted.

Thus, an improved method for combining assembly instruction generationwith sensor-based object detection and localization for efficient andautomatic identification of the at least one assembly part andlocalization of the at least one assembly part is provided.

In a preferred embodiment, the method comprises the step:

determining and/or adjusting parameters of the robot function whileexecuting the robot program.

Preferably, the robot program comprises parameters which can be adjustedwhile executing the robot program. When the at least one assembly partis identified and localized, the start pose, direction and/or pickingpose, which are relative poses, can be adjusted to absolute valuesindependent from another.

Thus, an improved method for combining assembly instruction generationwith sensor-based object detection and localization for efficient andautomatic identification of the at least one assembly part andlocalization of the at least one assembly part is provided.

According to an aspect, the system for robotic assembly, comprises adatabase, comprising product data, comprising product structure dataand/or product geometry data of a product to be assembled, a controlunit, configured for analysing the product data, thereby determiningrobot functions relating to functions of a robot for assembly of theproduct and generating a robot program comprising assembly instructionsdependent on the determined robot function and the product data and arobot system, comprising a robot controller, configured for executingthe generated robot program, thereby identifying and localizing the atleast one assembly part and assembling the product.

Thus, an improved system for combining assembly instruction generationwith sensor-based object detection and localization for efficient andautomatic identification of the at least one assembly part andlocalization of the at least one assembly part is provided.

In a preferred embodiment, the product data comprise CAD data of theproduct.

Thus, an improved system for combining CAD based assembly instructiongeneration with sensor-based object detection and localization forefficient and automatic identification of the at least one assembly partand localization of the at least one assembly part is provided.

In a preferred embodiment, the robot system comprises a RFID readersystem with at least one RFID reader, configured for identifying the atleast one assembly part of the product.

Thus, an improved identification of the at least one assembly part canbe provided.

In a preferred embodiment, the robot system comprises a vision system,preferably with at least one camera and/or a data processing unit,configured for absolutely localizing an assembly part of the product.

The camera preferably is a 3D camera.

Thus, an improved localization of the at least one assembly part can beprovided.

In a preferred embodiment, the at least one assembly part is providedwith a RFID transponder.

Each assembly part is provided with a RFID transponder. The RFIDtransponder is preferably disposed directly on the assembly part. TheRFID transponder alternatively is disposed on a container storing theassembly part and/or feeding system for feeding the assembly part. Everycontainer preferably contains only assembly parts of the same type.Containers and feeding system are especially suitable for relativelysmall assembly parts. The RFID transponder preferably stores at least apart identification numbers of the assembly part.

The RFID transponder can be identified by the RFID reader system inorder to identify the assembly part provided with the RFID transponder.

Thus, an improved identification of the at least one assembly part canbe provided.

The present invention also relates to a computer program productincluding computer program code for controlling one or more processorsof a device adapted to be connected to a communication network and/orconfigured to store a standardized configuration representation,assembly particularly, a computer program product including a computerreadable medium containing therein the computer program code.

Preferably, the functional modules and/or the configuration mechanismsare implemented as programmed software modules or procedures,respectively; however, one skilled in the art will understand that thefunctional modules and/or the configuration mechanisms can beimplemented fully or assembly partially in hardware.

FIG. 1 shows a system 10 for robot assembly comprising a database 20, acontrol unit 30 and a robot system 40. The control unit 30 comprises ananalysing unit 31 and a generation unit 32. The robot system 40comprises a RFID reader system 41, a vision system 42 and a robotcontroller 43. The database contains product data Dp of a product 60 tobe assembled. The product data Dp comprises product structure dataand/or product geometry data of the product 60. For example, the productdata Dp comprise CAD data of the product. The CAD data preferably relateto 3D models of the product and the members of the product. The productdata Dp is provided to the control unit 30, in particular the analysingunit 31 of the control unit 30.

The analysing unit 31 analyses the product data Dp. Thus, robotfunctions F relating to functions of robot for assembly of the productare determined by the analysing unit 31. Additionally, the members ofthe product 60, preferably a base component 61 and at least one assemblypart 62 can be determined from the product data Dp. The robot functionsF are provided to the generation unit 32, which is configured forautomatically generating a robot program P or a module of the robotprogram P. The robot program P is provided to the robot controller 43 ofthe robot system 40.

The robot controller 43 executes the provided robot program P. Therebythe at least one assembly part 62 is identified and localized. Theassembly part 62 is identified by the RFID reader system 41 andlocalized by the vision system 42. The RFID reader system 41 and thevisions system 42 are provided with the product data Dp from thedatabase 20. In particular, the layout and design of the workspace, astart pose of the assembly part 62 and/or the picking pose of theassembly part 62 for assembly of the product is provided to the RFIDreader system 41 and the vision system 42. Thus, the RFID reader system41 and the vision system 42 have a rough estimation of the location ofthe assembly part 62. The RFID reader system 41 identifies the assemblypart 62 and provides RFID data Dr relating to the identity of theassembly part 62 to the robot controller 43. The vision system 42localizes the assembly part 62 and provides vision data Dv relating tothe location of the assembly part 62 to the robot controller 43. Basedon the RFID data Dr, the vision data Dv and the robot program P, therobot 50 for the assembly of the product 60 can be controlled,preferably by robot instructions I provided by the robot controller 43.The robot program P starts with a set of parameters in accordance to theproduct data Dp. As soon as the RFID reader system 41 and the visionsystem 42 are activated and identify and/or localize the assembly part62, the parameters of the robot program P are updated based on the RFIDdata Dr and/or the vision data Dv. Additionally, the product data Dpfrom the database 20 can be adjusted based on the RFID data Dr and/orthe vision data Dv.

FIG. 2 shows a schematic view of a system 10 for robot assembly. Thesystem 10 comprises a RFID reader system 41, a vision system 42, a robot50 for assembly of a product 60, a computer 70 and a robot workspace 80.The robot 50 and the product 60 are disposed on the robot workspace 80.The product 60 comprises a base component 61 with two openings 61 a,which in this example are the assembly locations where the assemblyparts 62 need to be inserted to assemble the product 60. The product 60further comprises two assembly parts 62, which are configured as screws.The assembly of the product 60 comprises screwing the two assembly parts62 into the openings 61 a of the base component 61, respectively.

The assembly parts 62 are located at pre-defined locations or regionswithin the robot workspace 80. This information relates to the productdata Dp, which are provided by the computer 70.

The assembly parts 62 may be supplied to the robot workspace 80 manuallyand/or using a feeding system.

Each of the two assembly parts 62 are provided with a RFID transponder62 a. The RFID transponders 62 a are preferably disposed directly on theassembly part 62. However, the RFID transponders 62 a may also bedisposed on a container and/or feeding system for each type of assemblypart 62. This is especially suitable for relatively small assemblyparts. The RFID transponders 62 a store at least the part identificationnumbers of the assembly parts 62.

The RFID reader system 41 comprises at least one RFID reader 41 a. TheRFID reader system 41 is configured for reading at least part of theidentification numbers of the respective assembly parts 62. The RFIDreader system 41 can obtain further information, in particular productdata, from the database 20. The RFID reader system 41 is connected tothe robot controller 43.

The vision system 42 comprises at least one camera 42 a, in particular a3D camera, as well as a data processing unit for determining the visiondata Dv.

An exemplary method for robotic assembly comprises that a control unit30, in this case a robot program generation software, retrieves from adatabase 20 the product data Dp, to be assembled by the robot system 10,where this can be triggered by an upstream system, or by a humanoperator via a user interface. The control unit 20 analyses the productdata Dp and obtains at least the following information:

Firstly, the base component 61 is obtained. Therefore, a localcoordinate system is created, if not already existing in the productdata Dp obtained from the database 20.

Secondly, the assembly parts 62, to be inserted, in this case screwedinto the openings 61 a of the base component 61, is obtained. Therefore,a local coordinate system for each assembly part 62 is created, if notalready existing in the product data Dp obtained from the database 20. Astart pose and a direction for insertion relative to the base component61 is obtained. Optionally, also a picking pose, which is a tool poserelative to the assembly part 62, is obtained.

The control unit 30, in particular the generation unit 32, automaticallygenerates a robot program module P calling the following assemblyinstructions I. Finding the assembly parts 62, for example an assemblypart identification number and/or an assembly part location. If at leastthe base component 61 is found, adjusting a reference coordinate systemof the assembly part 62 to the base components 61 coordinate system. Ifassembly parts 62 to be inserted are found, adjusting a picking pose tothe actual pose of the assembly part 62. If the base component 61 isfound and at least one assembly part 62 to be inserted is found, pickingthe assembly part 62 with the corresponding pick pose. Inserting theassembly part 62 with the corresponding start pose and direction ofinsertion. Updating product data Dp in the database 20 and/or on theRFID transponders 62 a.

The robot controller 40 executes the generated robot program P, callingthe above mentioned assembly instructions I. Finding the assembly parts62 is done by the RFID reader system 41 that detects the presence ofassembly parts 62. RFID transponders 62 a are placed on the assemblyparts 62 and/or on container of assembly parts 62, when a set of smallassembly parts 62 are delivered together. At least an assembly partidentification number is stored on the RFID transponder 62 a. In case ofRFID transponders on a container, a number of assembly parts 62 and apattern for placing assembly parts 62 in the container are stored eitheron the RFID transponder 62 a or in the database 20.

If available, pre-defined locations of the assembly parts 62 or assemblypart containers are received from the database 20, for example aworkspace layout.

Multiple RFID readers 41 a with defined detection ranges may be placedat different positions to roughly locate the assembly parts 62, ifworkspace layout data is not available.

If necessary, the vision system 42 detects the location of the assemblyparts 62 and/or corrects, or in other words updates, the pre-definedproduct data Dp, in particular location data to match the realinstallation. Therefore, a CAD model of the assembly parts 62 isreceived from the database 20. A 3D image is matched to the CAD modelsof assembly parts 62. In case of assembly parts 62 fed with containers,the vision system 42 can also detect the presence of assembly parts 62within the container. The poses and reference coordinate system isadjusted accordingly using the above obtained part locations and therelative poses of the assembly parts 62 from the database 20.

The matching of the 3D image to the CAD models of the assembly parts 62can be done more efficiently by using sub-portions of the image only,knowing the rough position of the assembly parts 62 from the productdata Dp, in particular the CAD data and/or by using depth thresholds,and largely reduce a dimension of model matching when assembly parts 62are supplied in known orientations only.

FIG. 3 shows a flow chart of a method for robot assembly. In step S10,product data Dp is received, comprising product structure data and/orproduct geometry data of a product 60 with a base component 61 and atleast one assembly part 62 to be assembled. In step S20, the productdata Dp is analysed, thereby determining robot functions F relating tofunctions of a robot 50 for assembly of the product 60. In step S30, arobot program P comprising assembly instructions I dependent on thedetermined robot function F and the product data Dp is generated. Instep 40, the generated robot program P is executed, thereby identifyingand localizing the at least one assembly part 62 and assembling theproduct 60.

While the invention has been illustrated and described in detail in thedrawings and foregoing description, such illustration and descriptionare to be considered illustrative or exemplary and not restrictive. Itwill be understood that changes and modifications may be made by thoseof ordinary skill within the scope of the following claims. Inparticular, the present invention covers further embodiments with anycombination of features from different embodiments described above andbelow. Additionally, statements made herein characterizing the inventionrefer to an embodiment of the invention and not necessarily allembodiments.

The terms used in the claims should be construed to have the broadestreasonable interpretation consistent with the foregoing description. Forexample, the use of the article “a” or “the” in introducing an elementshould not be interpreted as being exclusive of a plurality of elements.Likewise, the recitation of “or” should be interpreted as beinginclusive, such that the recitation of “A or B” is not exclusive of “Aand B,” unless it is clear from the context or the foregoing descriptionthat only one of A and B is intended. Further, the recitation of “atleast one of A, B and C” should be interpreted as one or more of a groupof elements consisting of A, B and C, and should not be interpreted asrequiring at least one of each of the listed elements A, B and C,regardless of whether A, B and C are related as categories or otherwise.Moreover, the recitation of “A, B and/or C” or “at least one of A, B orC” should be interpreted as including any singular entity from thelisted elements, e.g., A, any subset from the listed elements, e.g., Aand B, or the entire list of elements A, B and C.

LIST OF REFERENCE SYMBOLS

-   10 system-   20 database-   30 control unit-   31 analysing unit-   32 generation unit-   40 robot system-   41 RFID reader system-   41 a RFID reader-   42 vision system-   42 a 3D camera-   43 robot controller-   50 robot-   60 product-   61 base component-   61 a opening-   62 assembly part-   62 a RFID tag-   70 computer-   80 workspace-   Dp product data-   Dr RFID data-   Dv vision data-   I assembly instructions-   P robot program-   F robot functions-   S10 receiving product data-   S20 analysing the product data-   S30 generating a robot program-   S40 executing the robot program

What is claimed is:
 1. A method for robotic assembly, comprising:receiving product data comprising product structure data and/or productgeometry data of a product with a base component and at least oneassembly part to be assembled; analyzing the product data to determinerobot functions relating to functions of a robot for assembly of theproduct as determined robot functions; generating a robot programcomprising assembly instructions dependent on the determined robotfunctions and the product data; and executing the generated robotprogram so as to identify and/or localize the at least one assembly partand assemble the product.
 2. The method of claim 1, further comprising:receiving product data from CAD data of the product.
 3. The method ofclaim 1, further comprising: identifying the at least one assembly partof the product by a RFID reader system while executing the robotprogram.
 4. The method of claim 1, further comprising: localizing the atleast one assembly part of the product by a vision system whileexecuting the robot program.
 5. The method of claim 4, furthercomprising: determining a CAD model of the at least one assembly partfrom the product data; and matching a 3D image of the vision system tothe determined CAD model of the at least one assembly part.
 6. Themethod of claim 1, wherein analyzing the product data comprises:creating a local coordinate system for the base component, creating alocal coordinate system for the at least one assembly part, determininga start pose of at least one assembly part, determining a direction ofthe at least one assembly part, and/or determining a picking pose of theat least one assembly part.
 7. The method of claim 6, wherein theassembly instructions comprise: finding the at least one assembly part,adjusting a reference coordinate system to the local coordinate systemof the base component, adjusting the picking pose of the at least oneassembly part, picking the at least one assembly part with thecorresponding picking pose, assembling the at least one assembly partwith the corresponding start pose and direction of the at least oneassembly part, and/or updating the product data.
 8. The method of claim1, further comprising: determining and/or adjusting parameters of therobot function while executing the robot program.
 9. A system forrobotic assembly, comprising: a database comprising product datacomprising product structure data and/or product geometry data of aproduct with a base component and at least one assembly part to beassembled; a control unit configured to analyze the product data so asto determine robot functions relating to functions of a robot forassembly of the product as determined robot functions and to generate arobot program comprising assembly instructions dependent on thedetermined robot functions and the product data as a generated robotprogram; and a robot system comprising a robot controller configured toexecute the generated robot program so as to identify and localize theat least one assembly part and to assemble the product.
 10. The systemof claim 9, wherein the product data comprise CAD data of the product.11. The system of claim 9, wherein the robot system further comprises aRFID reader system with at least one RFID reader configured to identifythe at least one assembly part of the product.
 12. The system of claim9, wherein the robot system further comprises a vision system configuredto localize the at least one assembly part of the product.
 13. Thesystem of claim 9, wherein the at least one assembly part is providedwith a RFID transponder.
 14. The system of claim 12, wherein the visionsystem comprises at least one camera and/or a data processing unit.